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This article introduces a high-speed distortionless predictive image-compression 
scheme that is based on differential pulse code modulation output modeling com- 
bined with efficient source-code design. Experimental resu/ts show that this scheme 
achieves compression that is very close to the difference entropy of the source. 


I. Introduction 

Digital data compression is the conversion of a stream 
of high-rate data (digital or analog) into a stream of rela- 
tively low-rate quantized data for communication over a 
digital communication link or storage in a digital memory. 
The goals are to reduce the volume of data for transmission 
over a digital channel and to archive to a digital medium. 
As the volume of speech and image data in the foreseeable 
future becomes prohibitively large for many communica- 
tion links or storage devices, the theory and practice of 
data compression are receiving increasing attention. 

Many data sources contain significant redundancy — 
symbol distribution, pattern repetition, and positional re- 
dundancy. Data compression schemes work by removing 
redundancy and encoding the new information. They can 
be broadly categorized into two classes: lossless and lossy . 
A lossless coder operating on a digital data source achieves 
compression without losing any of the digital source infor- 
mation, hence, the decoded image is no different from the 
original one. A lossy coder reduces the data rate by sac- 
rificing some information that is, or is thought to be, of 


little relevance to the user. As a result, the decoded data 
from a lossy coder show some coding noise, or error, when 
compared to the original data. 

Image data are a typical example of a data source with 
significant redundancy. Image-compression techniques are 
used to reduce the volume of raw image data, which is usu- 
ally too large for many communication links and storage 
devices. In the past 30 years, there has been a consid- 
erable amount of activity in image coding. Much of this 
research centers around the theme of using elegant com- 
pression algorithms to achieve a better compression ratio 
in the lossless compression scenario, and to achieve a better 
compression-distortion trade-off in the lossy compression 
scenario. Due to the recent advances in very large scale 
integration (VLSI) technology, it is now feasible to imple- 
ment these algorithms in practical situations, e.g., differen- 
tial pulse code modulation (DPCM) coding [1], runlength 
coding [2], the Rice machine [3], and arithmetic coding [4], 

DPCM schemes predict the present sample value based 
on the previous samples and send an encoded version of 
the difference between the predicted and actual values. 
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Because of its simplicity, compression efficiency, and ease 
of implementation, DPCM has become quite common in 
compressing speech and image data for transmission and 
storage. In digital processing, the input sequence to the 
DPCM scheme is generally digitized at the source itself 
by a fixed number of bits (typically 8 for images). Be- 
cause predictor values are required to be integer values, 
the DPCM output sequence will take integer values hav- 
ing a skewed probability distribution. The integer values 
can then be entropy-coded without distortion using an en- 
tropy coder such as the Huffman code [5] or the Rice al- 
gorithm [3]. A schematic diagram of a DPCM/entropy 
coding scheme is shown in Fig. 1. 

This article introduces a high-speed distortionless pre- 
dictive image-compression scheme that is based on DPCM 
output modeling and efficient source-code design. In gen- 
eral, this scheme is extremely simple to implement and, 
hence, has potential for high-rate, low-complexity appli- 
cations. This scheme is particularly useful for spacecraft 
imaging data, where exact reproduction is a requirement. 
Other uses include medical image data, where high speed 
and good quality are both extremely important. The re- 
mainder of this article is organized as follows: Section II 
describes two probabilistic models for the DPCM output 
integers and Section III outlines the compression algo- 
rithm that is based on the models proposed in Section II. 
Section IV describes the experimental results. Section V 
discusses the implementation issues. Section VI discusses 
the entropy and redundancy analysis for this scheme, Sec- 
tion VII addresses the source-model mismatch, and Sec- 
tion VIII gives the concluding remarks. 

II. DPCM Output Modeling 

This section describes two probabilistic models for the 
output samples of a simple DPCM scheme as shown in 
Fig. 1. To simplify the discussion that follows, a simple 
predictor consisting of a single delay is chosen. However, 
the results derived in this article are quite general and 
apply to DPCM coding schemes with other predictors. 

A. The Modified Lapiacian Model 

The Lapiacian distribution of the form 

Jl\ x ) = ~^2a eXP ' (0 

where pL denotes the mean and <t denotes the standard 
deviation, is traditionally used to model the unquantized 
output samples of a DPCM scheme [6,7]. By requiring the 


input sequence and the predictor output samples to be in- 
teger values, the DPCM output sequence will take integer 
values. The probability of the DPCM output integer i, 
which is denoted by p(i), can be evaluated by integrating 
from i — 0.5 to i + 0.5. In most well-behaved speech and 
image sources, fi is very close to zero. 

To simplify the discussion that follows, it is assumed 
that fi — 0. To verify the validity of the Lapiacian model, 
four images with 8-bit pixels are selected, and the afore- 
mentioned simple DPCM coding scheme is applied to en- 
code the images. For each image, the frequency distribu- 
tion of the DPCM output samples is tabulated and com- 
pared to the corresponding hypothetical distribution given 
in Eq. (1). The comparisons are shown in Fig. 2. The 
results were found to be unsatisfactory. The Lapiacian 
distribution was then modified by adjusting the standard 
deviation a to A <r such that \a = — 1 /(>/2 log( 1 - r(0)), 
and r(0) is the ratio of the number of zeros in the DPCM 
output stream to the total number of DPCM output sam- 
ples. The zero-mean modified Lapiacian distribution is 
given in Eq. (2) as 

r / \ 1 — n/2|x | 

fML(x) = ^=— exp ** (2) 

The introduction of A corresponds to adjusting the 
standard deviation a of the hypothetical distribution by 
matching the frequency of occurrences of the most prob- 
able event in the experiment, namely the integer zero, to 
the frequency of occurrences of zero predicted by the hy- 
pothetical Lapiacian model. The comparisons between the 
experimental frequency distribution and the hypothetical 
distribution of the images are shown in Fig. 3. The dif- 
ference entropy, defined as the entropy of the differences 
between adjacent pixels, is derived from the DPCM output 
samples and the entropy calculated from the hypothetical 
modified Lapiacian model of the images. The comparisons 
are shown in Table 1. This modified Lapiacian model can 
be used (1) to simplify the performance analysis of the 
DPCM scheme and (2) to design an optimal scalar quan- 
tizer (eg., a Lloyd-Max quantizer) for the DPCM output 
samples. 

B. The Two-Sided Geometric Model 

The above continuous modified Lapiacian distribution 
can be shown to provide a good foundation for efficient 
source-code design. However, this article deals with dis- 
crete (integer) DPCM outputs rather than a continuous 
random variable. The probability distribution of DPCM 
output integers derived from the modified Lapiacian distri- 
bution is geometric for all integers except zero. To circum- 
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vent these problems, a simple single-parameter discrete 
probability distribution for the DPCM output integers is 
introduced: 

PG(i) = Vi (3) 

where 

1 - r(0) 

1 + r(0) 

As in the modified Laplacian model introduced in Sec- 
tion II. A, the frequency of occurrences of the most prob- 
able event in the experiment, namely the integer zero, 
matches that predicted by the hypothetical two-sided geo- 
metric model. The comparisons between the experimental 
frequency distribution and the hypothetical distribution 
of the images are shown in Fig. 4. The two distributions 
are almost indistinguishable. The difference entropies de- 
rived from the hypothetical two-sided geometric model of 
the images are shown in Table 1. The two-sided geomet- 
ric model will be used in later sections to derive a simple 
and efficient lossless coding scheme for the DPCM output 
samples. 

III. Efficient Coding Based on the 
Two-Sided Geometric Model 

Constructing an optimal prefix code, by using the Huff- 
man algorithm for example, is quite a complex operation 
in hardware. This section presents efficient techniques to 
design a near-optimal prefix code to encode images with 
8 bits/pixel using the two-sided geometric model intro- 
duced in the previous section. In contrast, this code is very 
simple to implement in hardware. For most well-behaved 
images, frequency(i) « frequency ( — i) for i = 1 , 2 ,..., 255. 
Thus, in order to construct a code for the DPCM output 
samples, which have values ranging from —255 to 255, one 
can construct a prefix code for the integer set with val- 
ues from 0 to 255. An additional bit is then appended to 
each codeword, except the codewords representing 0, to 
indicate whether integer i or integer — i is sent. 

Gallager and van Voorhis presented an optimal binary 
prefix code for the set of geometrically distributed non- 
negative integers (single-sided geometric) [8]. This tech- 
nique is adapted and modified as discussed above to en- 
code the two-sided geometrically distributed DPCM out- 
put integers. The Gallager and van Voorhis optimal source 
code for geometrically distributed integer alphabets is a 
concatenation of a unary and a Huffman code; here it is 


named the Gallager-van Voorhis-Huffman (GVII) code. 
Let l be the integer satisfying 

0 l +0 I+l < 1 < 0 l +6 1 - 1 (4) 

where 

l-r(Q) 

1 + r(0) 

as defined in Section II. B. It is easy to see that for any 0 , 

0 < 0 < 1, there is a unique positive integer / satisfying 
Eq, (4). Let a non-negative number i be represented by 

1 = / j + r where j = the integer part of i/l, and 

r = [i] mod /. Gallager and van Voorhis showed that an 
optimal code for the non-negative integers is the concate- 
nation of a unary code that is used to encode j, and a 
Huffman code that is used to encode r, 0 < r < / — 1. 

Each integer r, 0 < r < l — 1, represents an equivalence 
class modulo 1. Gallager and van Voorhis showed that the 
integer set {r:0<r</ — 1} has a distribution 


and the sum of the two least likely letters exceeds the 
probability of the most likely. The length of the optimal 
codewords can differ by at most 1. It can be shown that 
the optimal coding for this integer set is to use codewords 
of length |_log 2 /J for i < 2^ og3/+1 J - /, and codewords of 
length [log 2 l\ + I otherwise. 

In this article, a simple construction is proposed to gen- 
erate a Huffman code for the integer set {r : 0 < r < /— 1}. 
The construction algorithm is as follows: 

(1) Generate the preliminary list L of 2t- log2/ J binary 
sequences {00 . . . 0, . . . , 1 1 . . . 1}, each of which has 
length |k>g 2 /J. 

(2) Append to each of the last / — 2^ log3/ J binary se- 
quences in L either a 0 or a 1 to generate two binary 
sequences of length U °&2 JJ + li and call the new list 
L'. 

It is not hard to see that V has a list of / prefix- 
conditioned codewords, with 2L log2 * +1 J — / codewords of 
length [log 2 /J, and the rest of length [\og 2 /J + 1; V is an 
optimal Huffman code for{r:0<r</— 1). Thus, each 
DPCM output integer i can be efficiently encoded using 
a concatenation of a unary code, a Huffman code, and a 
sign bit (except for * = 0). 
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IV. Experimental Results 

A number of images with 8-bit pixels were selected, 
including both planetary and nonplanetary images. The 
difference entropy of these images was evaluated, and the 
images were compressed using the Rice algorithm and the 
DPCM/GVH code. The difference entropy and the av- 
erage numbers of bits/pixel used to encode the images 
using the Rice algorithm and the DPCM/GVH code are 
shown in Table 2. Experimental results show that in all 
cases the DPCM/GVH scheme achieves compression that 
is very close to the difference entropy of the sources, and 
the compression performances of the Rice algorithm and 
the DPCM/GVH code are almost indistinguishable. 

V. Some Implementation Issues of the 
DPCM/GVH Scheme 

Like the Rice algorithm [3], the DPCM/GVH code 
yields a performance (average number of bits per pixel) 
that approaches the difference entropy of the source. As 
shown in Sections III and VI, the mathematical theories 
required to derive this algorithm and to show its efficiency 
are quite elegant. Its implementation, however, is very 
simple. This section describes some practical implementa- 
tion issues of the DPCM/GVH coding scheme. 

As discussed in Section III, the decoding process re- 
quires finding a unique integer / such that 6 l + 0 /+1 < 1 
< 0 l + O 1 * 1 . The range of 0 , and thus r(0), that gives a 
certain / can be precalculated, and / can be found easily 
by using a table look-up on r( 0). The maximum value for 
/ is arbitrary; here it is chosen to be 16. Table 3 gives 
the ranges of 7'(0) for / = 1, . . . , 16. Note that this table 
look-up on V s is optimally designed for the single-sided 
geometric distribution [5], and there is no guarantee that 
it is optimal for the two-sided geometrical distribution de- 
scribed in this article. A look-up table that minimizes the 
redundancy of the code for the two-sided geometric model 
can be constructed by direct search, and it is given in Ta- 
ble 4. This is discussed further in Section VI. Once l is 
established, the rest of the encoding is trivial. For each 
DPCM output integer —255 < i < 255, one expresses 
\i\ = Ij H- r. Since 0 < \i\ < 255, this operation of find- 
ing j and r given \i\ can be implemented using a table 
look-up on 256 entities. Since 1 < / < 16, there are 16 
tables. The total memory size required is 256 x 16 x 2 — 
8192 bytes. Once j and r are found, j is encoded using a 
unary code (a runlength code with j zeros followed by a 
1), and r, 0 < r < l — 1, is encoded using a trivial Huffman 
code with codewords of lengths [log 2 /J and [log 2 /J + 1. 
The Huffman codes corresponding to different /’ s can be 
generated easily, and they may be stored in a read-only 


memory (ROM). For / = 1, \i\ mod / = 0 for all |i| and 
no Huffman code is needed. For 2 < / < 16, the Huffman 
codes are stored in a ROM with only (2-b 16) x 15/2 = 
135 entities, each of which is at most 4 bits long. This 
is shown in Table 5. Finally, a 0 or 1 is appended to the 
concatenated codeword to indicate the sign of a nonzero 
i. Of course, it is obvious from Table 4 that some values 
of / in the range 1 < / < 16 will never be used. The to- 
tal memory size can be reduced further by omitting those 
entities that correspond to the unused Ts. 

In summary, this scheme involves first counting the 
number of occurrences of identical consecutive digits in a 
block (a line or the whole picture) of pixels. A table look- 
up operation is then performed to find /, which charac- 
terizes the subsequent encoding of the pixels in the block. 
Each pixel in the block is then encoded with one subtrac- 
tion operation (difference between the present pixel value 
and previous pixel value) and two table look-ups. 

This scheme can be modified to fit different needs. In- 
stead of encoding the whole image using one /, one can 
encode one line at a time. This has the following advan- 
tages: 

(1) It makes the encoding more adaptive to local statis- 
tics. 

(2) It reduces the buffer size (buffer for one line instead 
of the whole picture). 

(3) It allows on-the-fly encoding. 

(4) It complies with the fixed-line rate format. 

However, a potential disadvantage is that the estimate 
of 0 is based on few T er samples and may not be as robust as 
when the estimate is taken over a large number of samples. 
As the number of samples used n is reduced, the possibility 
of having a mismatch between the model and the actual 
data is increased. 


VI. The Redundancy of Gallager- 
van Voorhis-Huffman Codes 

It has been demonstrated empirically that the GVII 
coding scheme yields prefix codes whose average length 
is very close to the difference entropy of the source, i.e., 
whose redundancy is quite small. The performance of the 
GVH codes is now analyzed in more detail, deriving closed- 
form analytic expressions as a function of 0 for the redun- 
dancy, the mean codelength, and entropy of both single- 
sided and two-sided integer geometric distributions. 
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It is both instructive and convenient to first analyze the Hence, the result is 

redundancy for a single-sided integer geometric source, i.e., 

a source where the letters are labeled i, 0 < i < oo, where tt = /i — H(X\) 


p(i) = (1-0)0* 

The infinite source is treated (rather than the trun- 
cated source where 0 < i < 255), since the difference that 
results from the truncation is negligible. In Gallager and 
van Voorhis’s original paper [8], results were stated for the 
entropy, mean codelength, and redundancy of the GVH 
code for this source. Define the entropy of the DPCM 
source to be H(X i) (where X\ is the discrete random vari- 
able corresponding to the single-sided geometric source), 
the mean codelength of the GVH code for X\ to be / i, 
and the redundancy of said code to be r\. Furthermore, 
let ii(i) be the length of the ith codeword, as determined 
by the GVH procedure. As described in Section III, 


MO = ji + n 

where 

ji = m + 1 

and 

r. = [log 2 (/)J , if i mod / < - / 

= [log 2 (/)J + 1, if i mod / > 2L log ’ ( ' )+1 J - / 
where l is a function of 0 as described in Section III. 

Appendix A shows that the entropy of this source can 
be written as 


= L 1 o 82(0J + 1 + Y _ Ji 


H 2 (0) 
1 - 6 


Since this is an optimal prefix code, rq < 1; and indeed, 
for most values of 0, rq is quite small. 


Next, the topic of relevance to DPCM image coding is 
addressed, namely the coding of the two-sided geometric 
model introduced in Section II. B, where for each letter i, 
— oo < i < oo, and 


P2(») 


1 - 9 

1 + 0 


0|.'l 


Once again, for convenience the infinite source alphabet 
(rather than the truncated case) is treated: the truncated 
case differs by additive terms on the order of 0 255 , which is 
vanishingly small for all values of 0 encountered in practice. 
It has been found that for planetary images, the difference 
entropy tends to be in the range of 2.0 to 4.5 bits [3], which 
corresponds to 6 in the range from 0.3 to 0.8 bits. 

The GVH code for the double-sided model is defined 
as described earlier, i.e., for i ^ 0, an extra sign bit is 
appended to the equivalent codeword for a single-sided 
source. Define l 2 as the mean codelength of such a coding 
scheme, and r 2 as the resulting redundancy. If l 2 (i) is 
defined as the length of the codeword assigned to letter i 
by this scheme, then 


l 2 (i) = Mi) + !* i i 1 0 


H{ Xi) 


h 2 (0) 

1-0 


where H 2 (p) = -plog 2 (p) — (1 — P)l°g 2 (l — P) ^ le 
nary entropy function. It is also shown that the mean 
codelength 1 is 


fi = Li°g2(0J + 1 + i s'fi 


where k is defined as 


= MO), * = o 

First derive an expression for the mean codelength / 2 , 
as a function of 0 and l. Note that / need not strictly be a 
function of 0 as before, i.e., there is no guarantee that the 
condition described in Section III, namely that l uniquely 
satisfies 


1 +0 < 9~ l < 1 + - 


k - 2L lo S2(0+iJ _ i 


1 This result is different from that given in [Sf There appears to 
be a typographical error in their equation for h , as they have the 
term flog 2 (/)l instead of L lo g2(0J- 


leads to an optimal code for the two-sided source. Hence, 
one may treat the optimal selection of / as a separate prob- 
lem; the derivations to follow are quite valid for any choice 
of /. The question of how to find the /’s that minimize r 2 
will be discussed later. 


77 


h = 53 P2(*) / 2(») 

i=-oo 

i=oo 

= p 2 (0)/i(0) + ]>3 p 2 (i)(ji(i) + l) 

X = 1 

+ P2(*')( / i(l*l) + !) 


= 2 53p 2 (i)/ 1 (») + 1 - p 2 (0) - p 2 (0)/ 2 (0) 
i=0 

But by definition 

„ u\ - EliH 
p 1 + 0 

which leads to 


h — jT^^ 1 + 1 “ P2(0) - P 2 (0)^(0) 


Since, in general 


and 


^2(0) — 1 + Li°g 2 ( 0 J 


pM = 


and /1 has been derived earlier, one can write 


It is shown in Appendix B that the entropy of the two- 
sided source can be written as 


H(X 2 ) = 53 p 2 (i)h(i) 


= lo S2 


(£9- 


2glog 2 (fl) 

(l + 0 )(l- 0 ) 


where X 2 is the discrete random variable corresponding to 
the double-sided geometric source. Hence, one can write 
down a closed-form expression for the redundancy of the 
coding scheme as a function of 0 and /, namely 

r 2 = h - H(x 2 ) 


- 1 + L lo g 2 (0J + Y^r§ ( e + j~Toi 


- log2 


(£S) 


2^Iog 2 (0) 

(l + 0 )(l- 0 ) 


One can find the value of l that minimizes r2 for a given 
0 by minimizing the terms in r 2 that depend on /, namely 


m = ^ (rrjT 


The optimal l values (over all ranges of 9 of interest) 
can be found by direct search. Table 4 shows the ranges 
of 


r(0) = T 


1-0 


+ 1 


/a - TTe ( Llog2(/)J + 1 + i4^) 

+ 1 " IT? ~ IT? ^ + t lo «2(0J ) 

= 1 + Li°g 2 (/)j + yto { e + rbO 

Hence, it is seen that the mean codelength for the two- 
sided GVH coding scheme is quite similar in form to the 
one-sided GVH result. Clearly, however, the difference in 
the two forms may lead to different optimal values of the 
parameter / for fixed 0, i.e., and l 2 may be minimized 
by different values of / over certain ranges of 9. 


for which each value of / is optimal, 1 < / < 30. Note 
in particular that some values of / are not used in Table 4 
and that the ranges are different from the single-sided case 
(for small values of /) as given by Table 3. 

Using the optimal / values, one can plot r2 as a func- 
tion of entropy of the two-sided geometric source, with the 
results shown in Fig. 5. This characteristic of decreasing 
redundancy as a function of increasing source entropy is 
quite general and has been bounded by Smyth [9] for arbi- 
trary distributions. The redundancy of this coding scheme 
yields only a fraction of a bit of inefficiency for ranges of 
the source entropy of practical interest. The redundancy 
of the Huffman code as a function of the source entropy for 
the two-sided geometric model is also plotted in Fig. 5 for 
comparison. Clearly, the GVH code is near optimal over 
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all ranges of the source entropy and, from a practical view- 
point, the redundancy improvement by using a Huffman 
code over the GVTI code is negligible. 

VII. Source-Model Mismatch 

An obvious topic of interest for further investigation is 
the robustness of the coding procedure subject to devia- 
tions of the true source from the parametric model, i.e., 
one would like to know how the statistical variations that 
can be expected in practice will affect the actual number 
of bits sent back. The mean codelength for the model 
will not be exactly equal to the mean codelength trans- 
mitted for the data. Of course, as described earlier, ex- 
cellent empirical performance in the actual encoding of 
planetary images has been observed, but a more quan- 
titative understanding of the trade-offs involved is desir- 
able. Gilbert [10] looked at techniques for designing codes 
that avoid having very long codewords and, hence, mini- 
mize worst-case performance in the case of source-model 
mismatch. Longo and Galasso [11] looked at very gen- 
eral cases for the minimum deviations required before a 
probability distribution is “nearer” another optimal code. 
However, their bounds generally depend on the smallest 
differences between source probabilities and, as such, are 
not of practical use for DPCM models (where some of the 


source probabilities are very small). Since neither of these 
approaches is quite appropriate for this problem, there is 
considerable room for further work. For flight project ap- 
plications in particular, it will be very important to under- 
stand and quantify the robustness of this type of model- 
based coding prior to practical implementation. 

VIII. Conclusions 

The DPCM/GVII coding scheme described in this arti- 
cle effectively replaces the optimal Huffman coding scheme 
with a near-optimal look-up table operation for the special 
case of DPCM noiseless coding of image data. Thus, the 
scheme is eminently suitable for applications that require 
very high-rate noiseless coding and/or a very simple hard- 
ware implementation of the same. Hence, for example, 
future NASA flight projects such as the proposed Presi- 
dential Lunar-Mars Space Initiative 2 may find this scheme 
very attractive. In addition, very high-rate, ground-based 
data communication applications such as video conferenc- 
ing and medical imaging may also be suitable areas for 
applying this approach. 

2 E. C. Posner, “Implications of the Presidential Lunar and Mars 
Space Initiative for the DSN,” JPL IOM ECP 89-67 (internal docu- 
ment), Jet Propulsion Laboratory, Pasadena, California, February 
1990. 
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Table 3. Optimal / values for a single-sided geometric distribu- 
tion as a function of r 0 , the proportion of zeros in the 
difference-statistic histogram 

Table 1. Comparison of the difference entropies of the images, 
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0.036807 

0.033058 

10 





0.033058 

0.030397 

11 





0.030397 

0.027749 

12 

Table 2. Comparisons of the compression performances of the 

0.027749 

0.026167 

13 

Rice algorithm and the DPCM/GVH code in terms of bit/pixel 

0.026167 

0.024066 

14 





0.024066 

0.022495 

15 

Images 

Difference 

Rice 

DPCM/GVH 

0.022495 

0.021450 

16 

entropy 

algorithm 

code 

0.021450 

0.019888 

17 





0.019888 

0.018849 

18 

Saturn 6 

2.958 

3.030 

3.281 

0.018849 

0.017812 

19 

USC girl 

5.061 

5.102 

5.272 

0.017812 

0.017294 

20 

Lena 

5.673 

5.812 

6.010 

0.017294 

0.016260 

21 

Moon 

5.514 

5.663 

5.645 

0.016260 

0.015744 

22 

Air scene 

6.153 

6.268 

6.174 

0.015744 

0.015228 

23 

Peppers 

5.094 

5.162 

5.215 

0.015228 

0.014199 

24 

Baboon 

6.352 

6.339 

6.354 

0.014199 

0.013685 

25 

Mercury 

4.092 

4.234 

4.162 

0.013685 

0.013171 

26 





0.013171 

0.012658 

28 





0.012658 

0.012146 

29 
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Table 4. Optimal / values for a double-sided geometric distribu- 
tion as a function of r 0 » the proportion of zeros in the 
difference-statistic histogram 


Start of range r\ (0) 

End of range r 2 ( 0 ) 

Optimal l 

1.000000 

0.296176 

1 

0.296176 

0.140251 

2 

0.140251 

0.126126 

3 

0.126126 

0.077586 

4 

0.077586 

0.063264 

5 

0.063264 

0.055966 

7 

0.055966 

0.041124 

8 

0.041124 

0.036807 

9 

0.036807 

0.033058 

10 

0.033058 

0.030397 

11 

0.030397 

0.027749 

12 

0.027749 

0.026167 

15 

0.026167 

0.021450 

16 

0.021450 

0.019888 

17 

0.019888 

0.018849 

18 

0.018849 

0.017812 

19 

0.017812 

0.017294 

20 

0.017294 

0.016260 

21 

0.016260 

0.015744 

22 

0.015744 

0.015228 

23 

0.015228 

0.014199 

24 

0.014199 

0.013685 

25 

0.013685 

0.013171 

26 

0.013171 

0.012658 

28 

0.012658 

0.012146 

29 


Table 5. Codeword tables for 1 = 1,. . .,16 


Codewords Table number 

Codewords Table number 

; 

00 

01 

100 

101 6 

no 

in 

0 

10 3 

11 

00 

010 

Oil 

100 7 

101 
no 
in 

00 

01 4 

10 

11 

00 

01 

10 5 

no 

m 

000 

001 8 
010 
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PROBABILITY OF OCCURRENCE 
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JILITY OF OC 



















ENTROPY OF TWO-SIDED GEOMETRIC SOURCE, bits 


Fig. 5. Redundancy versus entropy for a two-sided geometric 
source. 
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Appendix A 

Mean Codelength of GVH Code for Single-Sided Geometric Distribution 


This Appendix gives an expression for the mean code- 
length of the GVH code for a single-sided geometric dis- 
tribution as defined in Section V. One has 

oo 

i -0 

* 2=00 

= (!-*)£ Mi 

i -0 

( i — oo i — oo \ 

E + E 0ir > 

i -0 i= 0 / 

= (i-«) (*E 0i (L7J + *) + E 0i (ljos*(oj + «<) ) 

where 


Looking at the first sum, 

i=oo 

^ = E^LyJ 

i=0 

= O( 0 ° + 0 1 + 0 2 + ... 0 '- 1 ) 

+ 1(0' + 0 ,+1 + 6 I+2 + ... 0 2 ' -1 ) 

+ 2 (0 2 l + 0 2,+1 + 0 2,+2 + ... 0 3 '- 1 ) 


+ ... 



and 


< 5 ; = 1 if i (mod l) > k 
= 0 otherwise 


£ = 2 L 1 o S 3 < / )+ 1 J - / 


and so one has 


Since 


y ioc x = — — o 0 < a < 1 
^ ! - a ‘ 


one gets 


q _ l - £| g' 

1 1-0 (1-0) 2 


/» = oo . 1=00 

*i = (i-*) E^'LjJ + E^ 


, *=0 


i=0 


+ fll L lo «2(0J + X^ 


*=0 


t = 0 


— Si -j- 5*2 + S 3 -f- *?4 


Clearly 


S2 + £3 — 


1 + L*og 2 (0J 

1 -0 


6 ‘ 

(l-0)(l-0‘) 


For the fourth sum, 64, one gets 


i=oo 

= E 0i6 < 

i -0 


= (0 k + O k+1 + ... + 0'- 1 ) 

+ ( 9 k +< + O k+,+l + ... + 0 2 '-i) 
+ ... 




Adding Si + 54, one gets 


Si + S4 


1 0* 
1-0 1 - 0 ' 


Hence 


h = |>g 2 (0J + 1 + 1 _ Qi 


where k is defined as 


B k 


/ 1 - 0 '-* 1 \ 
v 1-0 1-0'y 


k = 2 '- loga ^^ +1 -* - l 



Appendix B 

Entropy of a Two-Sided Geometric Source as a Function of o 





20Iog 2 (fl) 

( 1 -*)(! + *) 
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